Image processing apparatus and image processing method

ABSTRACT

The present invention provides an image processing apparatus capable of preventing deterioration of judgment accuracy when performing document matching process. To be more specific, when selecting feature points in the neighborhood of a target feature point, a predetermined number of feature points are selected after excluding a prespecified number of feature points in turn from the feature point nearest to the target feature point. For example, when the plurality of feature points are selected, the plurality of feature points, after excluding at least a feature point existing at a position nearest to the target feature point, are selected. Thereby, without increasing the amount of feature points for calculation of features (feature vectors), it is possible to prevent the deterioration of judgment accuracy.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image processing apparatus forjudging similarity between inputted document image data and image datastored in advance.

2. Description of the Related Art

In this kind of image processing apparatus, there has beenconventionally performed image processing for reading a document by ascanner and checking image data obtained by the reading against imagedata stored in advance to judge the similarity between the images.

For example, there are proposed a method of extracting a keyword fromimage data read by an OCR (optical character reader) and judging thesimilarity of the image on the basis of the extracted keyword, a methodof limiting an image to be targeted by similarity judgment only to arecord form image with ruled lines and extracting the characteristics ofthe ruled lines to judge the similarity of the image, a method ofreplacing character strings and the like in image data with points anddetermining the positional relationship among the points (featurepoints) as features to judge the similarity of the image, and the like.

International Publication WO2006/92957 (publication date: Sep. 8, 2006)discloses an image processing apparatus as described below. That is,connected parts of an image picked up by a digital camera or read by ascanner is regarded as word regions, and the centroids of the wordregions are determined as feature points. The feature points are used tocalculate a geometric invariant, and features are furthermore determinedfrom this geometric invariant. The features, indexes indicating thefeature points, and an index indicating the image are stored in a hashtable.

In performing retrieval, feature points, features and indexes indicatingthe feature points for a retrieval query (inputted image) are determinedin a similar process, and the hash table is accessed for performingvoting for stored document images.

In determining the features described above, n feature points nearest toa certain target feature point are selected. Then, m (m<n) featurepoints are further selected from among the selected n feature points,and d (d=m or smaller) feature points are extracted from among the mfeature points. For all combinations, the features related to the dfeature points are calculated.

In the above retrieval method disclosed in International PublicationWO2006/92957, however, when the centroid of a character is assumed to bea feature point, there may be a case where features calculated frompatterns with almost no positional variations in centroids, such as longEnglish words, agree with each other even if they are originallydifferent character strings. Therefore, there is a problem that theaccuracy of judgment of an image including a lot of such patternsdeteriorates.

In view of the above situation, the object of the present invention isto provide an image processing apparatus and an image processing methodcapable of preventing deterioration of judgment accuracy when featurepoints are determined from a document image to calculate features (hashvalue) with the use of the feature points.

SUMMARY OF THE INVENTION

In order to achieve the above object, in the present invention, whenfeature points are determined from a document image to calculatefeatures (feature vectors) (hash value) with the use of the featurepoints, a predetermined number of feature points are selected afterexcluding a prespecified number of feature points in turn from thefeature point nearest to a target feature point to retrieve thesimilarity of the image without deterioration of the judgment accuracy.

That is, the present invention is an image processing apparatus whichcompares features extracted from inputted document image data withfeatures of a reference image stored in advance to perform a judgmentprocess on whether the inputted document image data is similar to thereference image stored in advance, wherein, when points indicating localcharacteristics of the image of the inputted document image data, whichare determined from plurality of pixel values in the inputted documentimage data, are extracted and regarded as feature points, the featuresare values calculated on the basis of the positional relationship amongplurality of feature points selected around a certain target featurepoint; and the image processing apparatus comprises a neighbor pointselection section for, when selecting the plurality of feature points,selecting a predetermined number of feature points after excluding aprespecified number of feature points in turn from the feature pointnearest to the target feature point.

According to the above configuration, it is possible to refer to a widerrange of feature points without increasing the number of feature pointsused for calculation of features, and therefore, without increasing theamount of calculation and the amount of data. It is also possible, evenin the case where the shapes formed by feature points extracted fromdifferent image data are locally similar to each other, to calculate thefeatures as different values.

Accordingly, it is possible to prevent such a situation from occurringthat, since the features of feature points determined from differentcharacter strings agree with each other, they are erroneously judged toagree with each other, and the accuracy of the final document imagesimilarity judgment deteriorates.

For example, when the centroid of a character is regarded as a featurepoint, there may be a case where features calculated from patterns withalmost no positional variations in centroids, such as long Englishwords, agree with each other. The above characteristic is especiallyeffective in the case of identifying an image including a lot of suchpatterns.

Here, the “inputted document image data” means the following two kindsof image data: (i) image data inputted by reading a document by ascanner and (ii) electronic data created by inputting necessary items inan electronic data format with the use of a computer (software). Fromthe viewpoint of practical use, the following two kinds of data areconceivable: data obtained by reading data on paper by a scanner andelectronizing the data, and data which is immediately created aselectronic data.

The image forming apparatus of the present invention is characterized inbeing provided with the image processing apparatus according to any ofthe above descriptions and an image output section which forms an imageaccording to inputted image data on a recording material.

According to the above configuration, it is possible to refer to a widerrange of feature points without increasing the number of feature pointsused for calculation of features, and therefore, without increasing theamount of calculation and the amount of data, and it is possible toprevent such a situation from occurring that feature points extractedfrom different image data are judged to be such feature points that theshapes formed by the feature points are locally similar to each other,and the accuracy of the final document image similarity judgmentdeteriorates.

Accordingly, it in possible to prevent such a situation from occurringthat, since the features of feature points determined from differentcharacter strings agree with each other, they are erroneously judged toagree with each other, and the accuracy of the final document imagesimilarity judgment deteriorates.

The image processing method according to the present invention is animage processing method for comparing features extracted from inputteddocument image data and features of a reference image stored in advanceto perform judgment on whether the inputted document image data issimilar to the reference image stored in advance, wherein, when pointsindicating local characteristics of the image of the inputted documentimage data, which are determined from plurality of pixel values in theinputted document image data, are extracted and regarded as featurepoints, the features are values calculated on the basis of thepositional relationship among plurality of feature points selectedaround a certain target feature point; and when the plurality of featurepoints are selected, the plurality of feature points, excluding at leasta feature point existing at a position nearest to the target featurepoint, are selected.

According to the above configuration, it is possible to refer to a widerrange of feature points without increasing the number of points used forcalculation of features, and therefore, without increasing the amount ofcalculation and the amount of data, and it is possible, even in the casewhere the shapes formed by feature points extracted from different imagedata are locally similar to each other, to calculate the features asdifferent values.

Furthermore, a program for causing the image processing apparatusaccording to the present invention to operate is provided with a programfor causing a computer to function as each of the above means.

According to the above configuration, it is possible to prevent such asituation from occurring that features determined from differentcharacter strings happen to agree with each other and erroneously judgedto agree with each other. Therefore, it is possible to realize a processwith an improved matching accuracy by software.

Furthermore, as a recording medium according to the present invention, acomputer-readable recording medium in which the above program isrecorded can be also provided.

According to the above configuration, it is possible to control theoperations of an image reading apparatus and an image forming apparatus,by the program read from the recording medium. Furthermore, with the useof a computer, it is possible to realize the above image processingmethod on the computer.

As described above, according to the present invention, by extractingfeature points excluding a feature point near to a target feature point,and calculating a feature with the use of the feature points, it ispossible to retrieve the similarity of an image without deterioration ofthe judgment accuracy.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a configuration block diagram of a feature point calculationsection of an image processing apparatus which is an embodiment of thepresent invention;

FIG. 2 is a configuration block diagram of a document matching processsection of the image processing apparatus of the present invention;

FIG. 3 is a flowchart of a document matching process of the presentinvention;

FIG. 4 is a diagram showing the centroid of a connected component of acharacter;

FIG. 5 is a diagram showing a result of extraction of feature points ofthe present invention;

FIG. 6 is a diagram showing relationships among feature points and atarget feature point;

FIGS. 7( a) to 7(f) are diagrams showing an example in which one hashvalue is calculated from one feature point;

FIGS. 8( a) to 8(h) are diagrams showing another example in which onehash value is calculated from one feature point;

FIGS. 9( a) and 9(b) are diagrams showing a relationship between a hashvalue and an index indicating a document;

FIG. 10 is a diagram of relationship between a document and the numberof votes obtained, for illustrating the processing method by a votingprocess section of the present invention;

FIG. 11 is a configuration block diagram of a feature calculationsection of the present invention;

FIG. 12 is a configuration block diagram of a digital color copyingmachine to which the present invention is applied;

FIG. 13 is a configuration block diagram of an image reading apparatusto which the present invention is applied;

FIG. 14 is a diagram showing a hash table;

FIG. 15 is a configuration block diagram of a digital colormulti-function peripheral to which the present invention is applied;

FIGS. 16( a) and 16(b) are diagrams showing examples of differentcharacter strings with little positional variations in centroids assimulation;

FIG. 17 is a diagram showing an example of selection of the featurepoints of the present invention from character strings corresponding tothe character strings in FIG. 16; and

FIG. 18 is a diagram showing another example of selection of the featurepoints of the present invention from character strings corresponding tothe character strings in FIG. 16.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

An embodiment of the present invention will be described below on thebasis of drawings. FIG. 1 shows an example of the configuration of afeature point calculation section used in the present invention. FIG. 2is a configuration block diagram of a document matching process sectionused in the present invention. FIG. 3 is a flowchart of a documentmatching process.

First, description will be made on the configuration of the documentmatching process section which is an essentially characteristic portionof the present invention. A document matching process section 101 isconfigured by a feature point calculation section 102, a featurecalculation section 103, a voting process section 104, a similarityjudgment process section 105, a storage process section 106, a controlsection 107 and a memory 108.

The feature point calculation section 102 extracts a connected componentof a character string or a ruled line and calculates the centroid of theconnected component as a feature point 110.

FIG. 4 is a diagram showing the centroid 110 of a connected component ofan English alphabet letter “A”. FIG. 5 is a diagram showing a result ofextraction of feature points of a character string 111.

The feature calculation section 103 calculates an invariant which is notchanged by rotation, enlargement and reduction, that is, features whichare parameters not changed by geometrical change, including rotation,translation and scaling, of a document image, with the use of thefeature points 110 calculated by the feature point calculation section102. Feature points in the neighborhood of a target feature point areselected and used to calculate the features.

In the present invention, when the feature points are selected, aprespecified number of the nearest neighbor feature points are excluded,and then the feature points are selected. The reason and an example ofthe calculation method will be described later.

The voting process section 104 casts a vote for a document stored in ahash table with the use of the features (feature vectors) (hash values)calculated from the document.

FIG. 10 is a graph showing a relationship between a document and thenumber of votes obtained.

The similarity judgment process section 105 extracts a document (index)which has obtained the largest number of votes and the number of votesobtained, on the basis of a result of the voting process by the votingprocess section 104. Furthermore, the extracted number of votes obtainedis compared with a predetermined threshold to calculate similarity.Alternatively, the extracted number of votes obtained is divided by themaximum number of votes obtained by the document for normalization, andthe result is compared with a predetermined threshold. As an example ofthe threshold in this case, the threshold is set, for example, to 0.8 orabove. If there is a handwriting portion, the number of votes obtainedmay be larger than the maximum number of votes obtained. Therefore,there is a possibility that the similarity is larger than 1.

The maximum number of votes obtained is indicated by the number offeature points×the number of hash values calculated from one featurepoint (target feature point).

The examples in FIGS. 6 and 7 shows that one hash value is calculatedfrom one feature point as the simplest example. However, by changing themethod for selecting feature points around a target feature point,plurality of hash values can be calculated from one feature point. Forexample, the following method is conceivable. When six points areextracted as feature points around a target feature point, there are sixcombinations for extracting five points from the six points. For each ofthe six combinations, three points are extracted from five points todetermine an invariant, and hash values are calculated.

The storage process section 106 stores an index (document ID) indicatingthe document according to the features (feature vectors) (hash values)calculated from the document. When a storage process is performed, theprocesses by the voting process section 104 and the similarity judgmentprocess section 105 are skipped (not performed). On the contrary, in thecase where the document matching process is performed, the process bythe storage process section 106 is skipped.

The control section 107 is configured by the CPU of a microcomputer andcontrols access to the above process sections 102 to 106 and the memory108.

Next, the configuration of the feature point calculation section 102will be described on the basis of FIG. 1. The feature point calculationsection 102 is configured by a signal conversion section 112, aresolution conversion section 113, an MTF correction process section114, a binarization process section 115 and a centroid calculationsection 116.

The signal conversion section 112 is a process section for, wheninputted image data is a color image, achromatizing it and converting itto a Lightness or Luminance signal. For example, Luminance Y isdetermined from the following formula;Y _(j)=0.30R _(j)+0.59G _(j)+0.11B _(j)

Y_(j): Luminance value of each pixel; R_(j), G_(j), B_(j): colorcomponents of each pixel

Instead of using the above method, an RGB signal may be converted to aCIE1976 L*a*b* signal (CIE: Commission International de l'Eclairage; L*:Lightness; a*: redness-greenness; b*: yellowness-blueness; Chrominance).

The resolution conversion section 113 performs a process for, when themagnification of the inputted image data has been optically changed byan image input apparatus, changing the magnification of the data againso that the resolution becomes a predetermined resolution. Theresolution conversion section 113 is also used for resolution conversionfor reducing the resolution to be lower than the resolution at the timeof reading by an image input apparatus at the same magnification inorder to reduce the amount of processing at the subsequent stage. Forexample, image data read with 600 dpi (dot per inch) is converted to 300dpi data.

The MTF correction process section 114 performs a process for correctingthe transfer characteristic (MTF: Modulation Transfer Function) of anoptical pickup in an electronic circuit, and it is used to accommodatethe problem that the spatial frequency characteristic of an image inputapparatus differs according to the model.

As for an image signal outputted from a CCD, deterioration of the MTF(Modulation Transfer Function) is caused due to the opening degree ofthe aperture of optical parts, such as a lens and a mirror, and thereceiving surface of the CCD, transmission efficiency, image lags, anintegration effect and non-uniform operations of physical scanning, andthe like Consequently, a read image is blurred due to the deteriorationof the MTF.

The MTF correction process section performs a process for correcting theblurredness caused by deterioration of the MTF by performing a suitablefilter process (enhancement process). The MTF correction process sectionis also used to suppress a high-frequency component which is notrequired for a subsequent-stage feature point extraction process. Thatis, enhancement and smoothing processes are performed with the use of amixing filter (as for the filter coefficient, see FIG. 13).

By comparing the achromatized image data (a Luminance value (Luminancesignal) or a Lightness value (Lightness signal)) with a threshold, thebinarization process section 115 binarizes the image.

The centroid calculation section 116 performs labeling (a labelattaching process) for each pixel on the basis of the binarized imagedata (for example, indicated by “1” and “0”), identifies a connectedcomponent where pixels attached with the same label are connected,extracts the centroid 110 of the identified connected component as afeature point, and outputs the extracted feature point 110 to thefeature calculation section 103. The feature point 110 can be indicatedby coordinate values (x and y coordinates) in the binarized image.

Next, the configuration of the feature calculation section 103 and anexample of a feature calculation method will be shown. As shown in FIG.10, the feature calculation section 103 is provided with a neighborpoint selection section 117 which performs a process for selectingfeature points around a target feature point from among plurality offeature points extracted by the feature point calculation section 102,and a hash value calculation section 118 which calculates hash values(features) from the selected neighbor points.

An example of the feature calculation method will be shown below.

-   i) The feature point calculation section 102 calculates the centroid    (feature point) of a connected part of binarized image data. FIGS. 4    and 5 show an example in which a feature point of a character string    is calculated.-   ii) Feature points around a target feature point are selected (the    neighbor point selection section 117). As shown in FIG. 6, if    feature points around a target feature point a are denoted by b, c,    d, e and f, the four feature points b, c, d and e around the target    feature point a which are nearest to the target feature point a are    selected.-   iii) Three points are further selected from the selected four    feature points (the neighbor point selection section 117), and three    kinds (FIGS. 7(A) to 7(C); 7(D) to 7(F)) of invariant Hij are    determined. For example, the remainder obtained from the following    formula is regarded as a hash value Hi (features) (the hash value    calculation section 118).    Hi=(Hi1×10² +Hi2×10¹ +Hi3×10⁰)/D

D: constant determined by setting the maximum value of hash value

i: natural number

Alternatively, three points are further selected from the selected tourfeature points, and four kinds (FIGS. 3(A) to 8(D); 8(E) to 8(H)) ofinvariant Hij are determined. For example, the hash value Hi (features)may be determined from the following formula.Hi=(Hi1×10³ +Hi2×10² +Hi3×10¹ +Hi4×10⁰)/D

Here, as the formula for calculating the hash value, a well-known hashfunction other than the above may be used.

The invariant Hij is calculated, for example, from the followingformula:

${{Hij} = {\frac{Aij}{Bij}\mspace{11mu}{Aij}}};$Bij: distance between feature points

The respective distances between feature points with respect to Aij andBij are calculated from the coordinates of the feature points related tothe distances. Thereby, the value Hij is an invariant which is notchanged by similarity conversion.

-   iv) The hash value and an index (IDk) indicating the document are    stored in a hash table (see FIG. 9). If hash values are the same    value (H1=H5 in FIG. 9(B)), they may be integrated into one.

Next, a feature point selection method for calculating features will bedescribed. In the case of selecting, for example, four feature pointsaround a target feature point and further selecting three points fromamong the selected four feature points to determine three or four kindsof invariant, there may be a case where the same features are calculatedfor different character strings because positional variations incentroids are narrow and, therefore, the shapes formed by the featurepoints of the character strings are similar to each other.

For example, FIG. 16 shows an example of narrow positional variations incentroids, and FIGS. 16(A) and 16(B) show different character strings assimulation. In FIG. 16, a circle indicates a feature point, and a filledcircle indicates a feature point selected to calculate the features of atarget feature point.

As shown in FIGS. 16(A) and 16(B), though the feature points in the twofigures have been extracted from different character strings, they arejudged to agree with each other. Thus, the judgment accuracydeteriorates. Especially, this is remarkable in an English sentence andthe like.

Accordingly, in the present invention, when feature points around atarget feature point are selected, a predetermined number of featurepoints are selected after excluding a prespecified number of featurepoints in turn from the feature point nearest to the target featurepoint.

By selecting feature points away from a target feature point, thepossibility that feature points in the same shape are selected fromdifferent image patterns (character strings) can be reduced, and it ispossible to reduce the probability that the features of differentpatterns erroneously agree with each other.

FIG. 17 is a diagram showing a method of the present invention forselecting feature points corresponding to the character strings in FIGS.16( a) and 16(b). The figure shows an example in which, “1” is set as aspecified number of points when the feature points are selected, thatis, the nearest neighbor feature point is excluded. The figure shows anexample in which, though two character strings are different from eachother, the shapes formed by four feature points which have been selectedto calculate the features of a target feature point are similar to eachother and, therefore, the features of the determined target featurepoints of the two character strings agree with each other.

In comparison, the example in FIGS. 17( a) and 17(b) shows that, sincefour points excluding the nearest neighbor feature point are selected,the points positioned one point farther than the points in FIG. 16 arereferred to, and therefore, difference occurs between the shapes formedby the selected four points, and the feature points are differentvalues.

FIGS. 18( a) and 18(b) show an example in which, as the specified numberof points, the two nearest neighbor feature points as excluded. In thisexample, points positioned much farther than the points in FIG. 17 areused for calculation of the features of a target feature point.Therefore, the possibility that calculated features erroneously agreewith each other is reduced more.

Next, the entire document matching process will be described on thebasis of the flowchart in FIG. 3. First, it is determined whether astorage mode has been selected or not. If the storage is selected, aprocess for calculating feature points from inputted document image datais performed, and a process for calculating features from the calculatedfeature points is performed. Then, a process for storing the featuresand an index indicating the document with a table is performed, and thedocument matching process ends.

If the storage mode is not selected, the process for calculating featurepoints is performed first similarly to the case of the storage mode.Then, features are calculated from the calculated feature points andcompared with the features of a stored reference image to judge whetherthe inputted document image data is similar to the reference imagestored in advance. As a result of the judgment, if the data is similarto the reference image, it is determined whether a predetermined processis necessary or not. If it is necessary, a judgment signal “1” isoutputted, and the process ends. If the process is not necessary, or thedata is not similar to the reference image, a judgment signal “0” isoutputted, and the process ends.

In the present invention, when feature points around a target featurepoint are selected in calculation of features in the document matchingprocess, a predetermined number of feature points are selected afterexcluding a prespecified number of feature points in turn from thefeature point nearest to the target feature point. If the specifiednumber of points is, for example, “1”, feature points excluding thefeature point nearest to a target feature point, are selected. If thespecified number of points is “2”, a predetermined number of featurepoints, excluding the feature point nearest to a target feature pointand the second neighbor feature point, are selected.

By calculating features by selecting feature points as described above,it is possible to retrieve the similarity of an image withoutdeterioration of the judgment accuracy.

FIG. 12 is a configuration block diagram of a digital color copyingmachine as an image processing apparatus to which the present inventionis applied. A digital color copying machine 121 is configured by a colorimage input apparatus 122, a color image processing apparatus 123 and acolor image output apparatus 124.

The color image input apparatus 122 is configured, for example, by ascanner section provided with a device, for example a CCD, forconverting optical information to an electrical signal, and it outputs areflected light image from a document as an RGB analog signal.

The analog signal read by the color image input apparatus 122 is sentthrough an A/D conversion section 125, a shading correction section 126,a document matching process section 127, an input tone correctionsection 128, a segmentation process section 129, a color correctionsection 130, a black generation and under color removal section 131, aspatial filter process section 132, an output tone correction section133 and a tone reproduction process section 134 in that order within thecolor image processing apparatus 123, and it is outputted to a colorimage output apparatus 124 as C (cyan), M (magenta), Y (yellow) and K(black) signals.

The A/D conversion section 125 converts an RGB (R: red; G: green; B:blue) analog signal to a digital signal. The shading correction section126 performs a process for removing various distortions caused in theillumination system, image focusing system and image sensing system ofthe color image input apparatus 122 for the digital RGB signal sent fromthe A/D conversion section 125. Furthermore, the shading correctionsection 126 adjusts color balance, and performs a process for conversioninto a signal which can be easily processed by the color imageprocessing apparatus 123, such as a density (pixel value) signal.

The document matching process section 127 performs a feature pointcalculation process, and judges similarity between an inputted imagedata and reference document image data stored in advance with the use ofthe result of the feature point calculation process. If it is judgedthat there is similarity, processes such as copying, electronicdistribution, facsimile transmission and filing are prohibited as shownin FIG. 3. The document matching process section 127 outputs theinputted RGB signal to the input tone correction section 128 without anymodification.

The input tone correction section 128 performs an image qualityadjustment process such as removal of page background density andcontrast adjustment, for the RGB signal from which various distortionshave been removed by the shading correction section 126.

The segmentation process section 129 segments each pixel in inputtedimage data into any of a text area, a halftone dot area and a photograph(continuous tone) area, on the basis of the RGB signal. The segmentationprocess section 129 outputs, on the basis of the result of thesegmentation, a segmentation class signal indicating which area a pixelbelongs to, to the black generation and under color removal section 131,the spatial filter process section 132 and the tone reproduction processsection 134, and outputs an input signal outputted from the input tonecorrection section 128 to the subsequent-stage color correction section130 as it is.

The color correction section 130 performs a process for removing colorimpurity based on the spectroscopic characteristic of CMY color materialincluding a useless absorption component in order to realize faithfulcolor reproduction.

The black generation and under color removal section 131 performs aprocess for black generation for generating a black (K) signal from aCMY three-color signal after the color correction, and for generating anew CMY signal by subtracting the K signal obtained by the blackgeneration from the original CMY signal. Thereby, the CMY three-colorsignal is converted to a CMYK four-color signal.

The spatial filter process section 132 performs a spatial filter processby a digital filter for the image data of the CMYK signal inputted fromthe black generation and under color removal section 131, on the basisof the segmentation class signal, and corrects the spatial frequencycharacteristic. Thereby, it is possible to reduce blurredness andgraininess deterioration of an output image.

Similarly to the spatial filter process section 132, the tonereproduction process section 134 performs a predetermined process to bedescribed later, for the image data of the CMYK signal on the basis ofthe segmentation class signal.

For example, for an area segmented as a text by the segmentation processsection 129, the spatial filter process section 132 performs edgeenhancement processing so as to emphasize high-frequency components inorder to improve the reproducibility of texts. And the tone reproductionprocess section 134 performs a binarization or multi-level ditheringprocess using a high-resolution screen suitable for reproduction of ahigh-frequency component.

As for an area segmented as halftone dots by the segmentation processsection 129, the spatial filter process section 132 performs a low-passfilter process for removing an input halftone dot component.

Then, the output tone correction section 133 performs an output tonecorrection process for converting a signal such as a density signal to ahalftone screen area ratio, which is a characteristic value of the colorimage output apparatus 124. After that, the tone reproduction processsection 134 performs a tone reproduction process so that, finally,separating the image into pixels and each tone of the pixels can bereproduced. As for an area segmented as a photograph by the segmentationprocess section 129, a binarization or multi-level dithering process isperformed with the use of a screen suitable for tone reproduction.

The image data for which each of the processes described above has beenperformed is stored in the storage device once, and, at a predeterminedtiming, it is read and inputted into the color image output apparatus.

The color image output apparatus 124 outputs image data on a recordingmedium such as paper. For example, a color image output apparatus usingan electrophotography system or an inkjet system can be given. However,the color image output apparatus 124 is not limited thereto. Theprocesses described above are controlled by a CPU (Central ProcessingUnit) not shown.

The present invention may be applied not to a digital copying machinebut to a digital color multi-function peripheral 141 (see FIG. 15) whichis provided with a copying function, a printer function, a facsimiletransmission function, a scan to e-mail function and the like.

As shown in FIG. 15, the digital color multi-function peripheral 141 isfurther provided with a communication device 142 configured, forexample, by a modem or a network card. In the case of performingfacsimile transmission, when a transmission procedure with a destinationis performed via the modem and a transmittable state is secured, imagedata (image data read by a scanner) which has been compressed in apredetermined format is read from a memory. Then, a process such asconversion of the compression format are performed, and the image datais sequentially transmitted to the destination via a communication line.

In the case of receiving image data by facsimile, the CPU receives theimage data transmitted from an originating communication device andinputs it into the color image processing apparatus 123 while performinga communication procedure. In the color image processing apparatus 123,a compression/decompression process section (not shown) performs acompression/decompression process for the received image data. Arotation process and a resolution conversion process are performed forthe decompressed image data as necessary. Then, output tone correctionand tone reproduction processes are performed, and the image data isoutputted from the color image output apparatus 124.

It is also possible to perform data communication with a computer oranother digital multi-function peripheral connected to the network via anetwork card or a LAN cable.

Though the color multi-function peripheral 141 has been described above,a monochrome multi-function peripheral is also possible. Furthermore, asingle facsimile communication apparatus is also possible.

FIG. 13 is a block diagram showing the configuration of an image readingapparatus (flat bed scanner) which is provided with the documentmatching process section constituting the essentially characteristicportion of the present invention. As shown in the figure, a color imageprocessing apparatus 123 is configured by an A/D conversion section 125,a shading correction section 126 and a document matching process section127. By further connecting a color image input apparatus 122 to thecolor image processing apparatus 123, an image reading apparatus 151 asa whole is configured.

The color image input apparatus (image reading section) 122 isconfigured, for example, by a scanner section provided with a CCD(Charge Coupled Device), and it reads a reflected light image from adocument as an RGB (R: red; G: green; B: blue) analog signal by the CCDand inputs it into the color image processing apparatus.

The analog signal read by the color image input apparatus 122 is sentthrough the A/D conversion section 125, the shading correction section126 and the document matching process section 127 in that order withinthe color image processing apparatus 123.

The A/D (analog/digital) conversion section 125 converts an RGB analogsignal to a digital signal. The shading correction section 126 performsa process for removing various distortions caused in the illuminationsystem, image focusing system and image sensing system of the colorimage input apparatus 122 for the digital RGB signal sent from the A/Dconversion section 125. The shading correction section 126 adjusts colorbalance. The shading correction section 126 also converts an RGBreflectance signal to a density signal.

The document matching process section 127 performs a feature pointcalculation process, and judges similarity between an inputted imagedata and reference document image data stored in advance with the use ofthe result of the feature point calculation process. If it is judgedthat there is similarity, a judgment signal which prohibits processessuch as copying, electronic distribution, facsimile transmission andfiling is outputted.

The judgment signal is transmitted to a printer or a multi-functionperipheral together with read image data via a network, or it isinputted to a printer via a computer or directly to a printer. In thiscase, it is necessary that the printer, the multi-function peripheral orthe computer can judge a signal indicating the contents of the process.It is also possible to output features calculated by the documentmatching process section 127 instead of the judgment signal.

A digital camera may be used as the image reading apparatus of thisembodiment.

Furthermore, according to the present invention, an image processingmethod for performing the document matching and output control describedabove can be recorded in a computer-readable recording medium in whichprogram codes of programs (an executable program, an intermediate codeprogram and a source program) to be executed by a computer are recorded.

Consequently, it is possible to portably provide a recording medium inwhich program codes for implementing an image processing method forperforming document matching, output control and document image storageprocesses are recorded.

As for the recording medium in this embodiment, since the processes areperformed by a microcomputer, a memory not shown, such as a ROM, itselfmay be a program medium. Such a program medium is also possible as canbe read by inserting a recording medium into a program reading medium asan external storage apparatus though it is not shown.

In any case, the stored program may be configured so as to be accessedand executed by a microprocessor. Alternatively, in any case, it ispossible that a program code is read, and the read program code isdownloaded to a program storage area (not shown) of a microcomputer andexecuted. The program to be downloaded is assumed to be stored in thebody apparatus in advance.

The above program medium is a recording medium configured in a mannerthat it can be separated from the body, and it may be a tape system,such as a magnetic tape and a cassette tape, a disk system such as amagnetic disk including a floppy disk and a hard disk, and an opticaldisk including a CD-ROM, an MO, an MD, and a DVD, a card system such asan IC card (including a memory card) and an optical card, or a mediumfor holding the program fixed therein, including a semiconductor memorysuch as a mask ROM, an EPROM (Erasable Programmable Read Only Memory),an EEPROM (Electrically Erasable Programmable Read Only Memory) and aflash ROM.

In this embodiment, since the system configuration is such that acommunication network including the Internet can be connected, themedium may be such that fluidly holds the program so that the programcode can be downloaded from the communication network.

In the case of downloading the program code from the communicationnetwork as described above, the program to be downloaded may be storedin the body apparatus in advance or may be installed from a differentrecording medium.

The present invention can be also realized in a form of a computer datasignal embedded in a carrier wave in which the program code is embodiedby electronic transmission.

The image processing method described above is executed by the recordingmedium being read by a program reading apparatus provided for a digitalcolor image forming apparatus or a computer system.

The computer system is configured by an image input apparatus such as aflat bed scanner, a film scanner and a digital camera, a computer onwhich various processes such as the above image processing method areperformed by a predetermined program being loaded thereto, an imagedisplay apparatus, such as a CRT display and a liquid crystal display,for displaying the result of the processes by the computer, and aprinter for outputting the result of the processes by the computer onpaper or the like. Furthermore, there is provided a network card or amodem as communication means for connecting to a server and the like viaa network.

As described above, according to the present invention, by extractingfeature points, excluding a prespecified number of feature points nearto a target feature point, and calculating features with the use of thefeature points, it is possible to retrieve the similarity of an imagewithout deterioration of the judgment accuracy.

1. An image processing apparatus which compares features extracted frominputted document image data with features of a reference image storedin advance to perform a judgment process on whether the inputteddocument image data is similar to the reference image stored in advance,wherein when points indicating local characteristics of the image of theinputted document image data, which are determined from plurality ofpixel values in the inputted document image data, are extracted andregarded as feature points, the features are values calculated on thebasis of the positional relationship among plurality of feature pointsselected around a certain target feature point; and the image processingapparatus comprises a neighbor point selection section for, whenselecting the plurality of feature points, selecting a predeterminednumber of feature points after excluding a prespecified number offeature points in turn from the feature point nearest to the targetfeature point.
 2. The image processing apparatus according to claim 1,wherein, when selecting the plurality of feature points, the neighborpoint selection section selects the plurality of feature points,excluding at least a feature point existing at a position nearest to thetarget feature point.
 3. The image processing apparatus according toclaim 1, wherein, when selecting the multiple feature points, theneighbor point selection section selects the multiple feature points,excluding at least a feature point existing at a position nearestneighbor to the target feature point and a feature point existing at thesecond neighbor position.
 4. An image forming apparatus comprising: theimage processing apparatus according to claim 1; and an image outputsection which forms an image according to inputted image data on arecording material.
 5. A computer-readable recording medium in which aprogram for causing the image processing apparatus according to claim 1to operate is recorded, the program being for causing a computer tofunction as the neighbor point selection section.
 6. An image processingmethod for comparing features extracted from inputted document imagedata with features of a reference image stored in advance to performjudgment on whether the inputted document image data is similar to thereference image stored in advance, wherein when points indicating localcharacteristics of the image of the inputted document image data, whichare determined from plurality of pixel values in the inputted documentimage data, are extracted and regarded as feature points, the featuresare values calculated on basis of the positional relationship amongplurality of feature points selected around a certain target featurepoint; and when the plurality of feature points are selected, apredetermined number of feature points are selected after excluding aprespecified number of feature points in turn from the feature pointnearest to the target feature point.